Quantum Phase Transitions beyond the Landau's Paradigm in Sp(4) Spin System 
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We propose quantum phase transitions beyond the Landau's paradigm of Sp(4) spin Heisenberg 
models on the triangular and square lattices, motivated by the exact Sp(4)~ SO(5) symmetry of 
spin-3/2 fermionic cold atomic system with only wave scattering. On the triangular lattice, we 
study a phase transition between the v3 x y3 spin ordered phase and a Z2 spin liquid phase, this 
phase transition is described by an 0(8) sigma model in terms of fractionalized spinon fields, with 
significant anomalous scaling dimensions of spin order parameters. On the square lattice, we propose 
a deconfined critical point between the Neel order and the VBS order, which is described by the 
CP(3) model, and the monopole effect of the compact U(l) gauge field is expected to be suppressed 
at the critical point. 
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I. INTRODUCTION 



Landau's classic phase transition paradigm describes 
continuous phase transitions by symmetry breaking of 
the system^, and the powerful renormalization group 
theory developed by Wilson suffices this paradigm with 
systematic calculation techniques. Based on Landau- 
Ginzburg- Wilson (LGW) theory^, the continuous phase 
transition should be described by fluctuations of physical 
order parameters. A few years ago, it was proposed that 
a direct unfine-tuned continuous transition between two 
ordered phases which break different symmetries is pos- 
sible in quantum magnet^, which is forbidden in Lan- 
dau's theory. Recent numerical results suggest that this 
transition may exist in a SU(2) spin- 1/2 model with both 
Heisenberg and ring exchange^. The key feature of this 
nonlandau critical behavior is that at the critical point 
the field theory in terms of fractionalized objects with no 
obvious physical probe is a more appropriate description. 
In spite of the difficulty of probing the fractionalized ex- 
citations, the fractionalized nature of the critical point 
leads to enormous anomalous dimension of the physical 
order parameter distinct from the Wilson-Fisher fixed 
point or the mean field result, which can be checked ex- 
perimentally. 

In a seminal paper, it was proved that in spin-3/2 cold 
atom systems, with the standard s— wave scattering ap- 
proximation, the four-component spin-3/2 fermion multi- 
plet enjoys an enlarged Sp(4)~ SO(5) symmetry without 
fine-tuning any parameter—. By tuning the spin-0 and 
spin-2 scattering channels, there is one point with an even 
larger SU(4)d Sp(4) symmetryi&2. The fundamental 
representation of the 15 generators of SU(4) Lie-algebra 
can be divided into two groups: r a with a = 1, 2 • ■ ■ 5 
and r ab = i[r a ,r b ], and T a obey the Clifford algebra: 
rpa p6| _ 2S a },. Let us denote the fermion atom opera- 
tor as i/j a , then the fermion bilinear T a = tp^Tatp form a 
vector representation of Sp(4) group, and T a i, = tp^Tabt/j 
form an adjoint representation of Sp(4) group. In the 



particular representation we choose, 



1,2,3, r 4 = i®(i x ,r 5 = 1®/^. (i) 



The difference between SU(4) algebra and Sp(4) alge- 
bra is that, two Sp(4) particles can form a Sp(4) singlet 
through a 4 x 4 antisymmetric matrix J = ia v ® fi x , 
which satisfies the following algebra 



J 1 = -J, J 2 = -1, 



JTabJ = r* b , JT a J = -T a . (2) 



One can see that JapipaV^p creates a Sp(4) invariant 
state, therefore the Valence Bond Solid (VBS) state of 
SU(2) spin systems can be naturally generalized to Sp(4) 
spin systems. By contrast, two SU(4) particles can only 
form a 6 dimensional representation and a 10 dimensional 
representation of SU(4) algebra, and the smallest SU(4) 
singlet always involves four particles. 

If we consider a Mott-Insulator phase of spin-3/2 cold 
atoms on the optical lattice with one particle per well 
on average, the effective spin Hamiltonian should be in- 
variant under Sp(4) transformations. The most general 
Sp(4)-Heisenberg model contains two terms: 



H = 



Jit f if - J 2 f ?f j. 



(3) 



<»,j> 



The key difference between T a b and T a is their be- 
havior under time-reversal transformation. The time- 
reversal transformation on the fermion multiplet ip a is 
ipa ~ * Ja^^i this implies that T a i, (T a ) is odd (even) 
under time-reversal. Also, if rewritten in terms of the 
original SU(2) spin-3/2 matrices, r a f, only involves the 
odd powers of spins, and T a only involves the even powers 
of spinsS. This model can be exactly realized in spin-3/2 
cold atom systems, the coefficients J\ and J2 are deter- 
mined by the spin-0 and spin-2 scattering parameters-. 
Clearly when — J2 = J\ the system has SU(4) symme- 
try. In this work we will consider the Heisenberg model 
on the triangular and square lattice, in the parameter 
regime with J± > 0. Our focus in the current work will 
be the nonlandau like quantum phase transitions, which 
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is also a larger spin generalization of the deconfined crit- 
ically discussed before. A more detailed analysis of the 
whole phase diagram of the Sp(4) Heisenberg model in 
© will be given in a future workiS. 

II. THE SP(4) HEISENBERG MODEL ON THE 
TRIANGULAR LATTICE 

Let us study the triangular lattice first, and we will 
use the standard Schwinger boson formalism to study 
the magnetic ordered phase. We introduce Schwinger 
boson spinon b a as usual Sf = b\ S*pbi t p, S a are the 15 
generators of SU(4) algebra in the fundamental represen- 
tation. This definition of spinon b a is subject to a local 
constraint: Yla=i b\ a bi,a = 1, which also manifests itself 
as a local U(l) degree of freedom: bi i0l — ► exp(z0 i )6 ijQ . 
Using the following identities^: 

T^r^ = 2<5 Q(T <5/3 7 + 2J al Jp a — SapSja, (4) 

the Hamiltonian (|3|) can be rewritten as 

H= J2 2(J 1 -J 2 )K} j k ij -2(J 1 + J 2 )Ql j Q ij , 

<i,j> 

Ki] = b ia bj, a , Qij = Japb^abj^p. (5) 

Now we introduce two variational parameters Kij = 
(Kij) and Qij = (Qij), and assuming these variational 
parameters are uniform on the whole lattice, we mean- 
field Hamiltonian for © reads: 

H mf = 2( Ji - J2)Kk t] - 2(Ji + J 2 )QQ„ + H.c. 

<i,j> 

-2(Ji - J 2 )K 2 + 2(J ± + J 2 )Q 2 - m(&1,A« - 1).(6) 

The following formalism is similar to reference^, which 
studied the SU(2) spin models on the triangular lat- 
tice. The term involving fi imposes the constraint on 
the Hilbert space of spinon JD a=1 b\ a bi ;Cl — 1. If the 
spectrum of the spinons is gapless, the spinon will con- 
dense at the minima of the Brillouin zone. By solving 
the self-consistent equations for K, Q and fi, we obtain 
that when J 2 /Ji > —0.3, there is a finite percentage of 
spinon condensate at momenta ±<fo = ~^)? which 

are the corners of the Brillouin Zone. The condensate 
density as a function of 3 2 j J\ is plotted in Fig. [TJ 

The gauge field fluctuation rooted in the constraint 
y^ a — i bj a bj.g = 1 is the most important correction to 
the mean field calculation above. The local constraint 
would in general induce U(l) gauge fluctuations. How- 
ever, the condensate obtained from the Schwinger boson 
formalism corresponds to the state with nonzero expecta- 
tion value Q = (Qij), which is a pairing amplitude. The 
pairing between nearest neighbor sites breaks the U(l) 
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FIG. 1: Mean field solutions on triangular lattice at different 
J2/J1 with Ji > 0. The y axes k c shows the density of spinon 
condensate, which is also proportional to \z\ 2 defined in Eq. 
[7] k c decreases to nearly zero (0.007) as J2/J1 decreases. 

gauge symmetry down to Z 2 gauge symmetry, therefore 
the long wavelength field theory of this condensate should 
only have Z 2 gauge symmetry. To understand this order, 
we define slow mode z a as 

b a (x) = e l ^ s z a (x) + e-^- s J aP z p {x), (7) 

now one can rewrite spin operators T a b and r a in terms 
of slow mode Zq clS 

f a6 ~ e i2 ^ s zJT ab z + H.c, 
r a — z'T a z = n a . (8) 

Therefore T a has a uniform order n a , while T ab is only 
ordered at finite momentum ±2<f - For completeness, one 
can define Sp(4) adjoint vector ni )ao and n 2t ab as 

ni.ab = Re[zJT ab z], n 2)0 6 = 1m[zJT ab z], (9) 

the order of T a b can be written in terms of ni jab and 
n 2 ,ab'- 

t ab ~ cos(2<f • x)ni tab + sin(2g • x)n 2 , ab , 

^ ni, ab n 2 , ab = 0. (10) 

a . 6 

ni ta b and n 2 ^ a b are two Sp(4) adjoint vectors "perpendic- 
ular" to each other. Since r a is time-reversal even, while 
T ab is time-reversal odd^, the condensate of z a has both 
uniform spin nematic order and y/3 x v3 order. 

The U(l) local gauge degree of freedom is lost in Eq. 
The residual gauge symmetry is only Z 2 which transforms 
z — > —z. Physically this implies that an arbitrary U(l) 
transformation of z field will result in a rotation of spin 
order parameter T a b- This situation is very similar to the 
spinon description of the v3 x y3 order of SU(2) spins 
on the triangular lattice^. The field theory describing 
this condensate should contain Z 2 gauge field. However, 
since Z 2 gauge field does not introduce any long range 
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interaction or critical behavior, we can safely integrate 
out the Z 2 gauge field. The field theory can then be 
written as 

L = \d,z\ 2 + r\z\ 2 +g(\z\ 2 ) 2 + --- (11) 

The ellipses include all the Sp(4) invariant terms. 

Apparently, without the ellipses, the Lagrangian (fTT]) 
enjoys an enlarged 0(8) symmetry once we define real bo- 
son field multiplet <p as (f> — (Re[zi], Im[zi], • • • , Im[z4])*, 
and the Lagrangian (jlip can be rewritten as 

8 

L = Y,(W 2 + r \^\ 2 + 9{\$\ 2 f + ■■■ (12) 

Q = l 

The Lagrangian (|12[) without other perturbations de- 
scribes an 0(8) transition, and the ordered state has 
ground state manifold (GSM) 

U(4)/[U(3) <g> Z 2 ] = S 7 /Z 2 = RP(7), (13) 

we mod Z 2 from S 7 because of the Z 2 gauge symmetry 
of z. There are certainly other terms in the field the- 
ory which can break the 0(8) symmetry down to Sp(4) 
symmetry, but all the terms allowed by Sp(4) symmetry 
and lattice symmetry include at least two derivatives, for 
instance ^apZad^z^ 2 . These terms change the Gold- 
stone mode dispersion but do not change the GSM, and 
since they contain high powers of z and also at least 
two derivatives, they are irrelevant at the 0(8) critical 
point. Other Sp(4) invariant terms without derivatives 

like J2a,b( n l,ab) 2 , Da("a) 2 , e abcdelll : a&™l ,cd« e et al. ei- 
ther vanish or can be rewritten in terms of powers of z'z, 
which preserves the 0(8) symmetry. Therefore we con- 
clude that the ground state manifold of the condensate 
is S 7 /Z 2 , and the transition between the condensate and 
disordered state by tuning J\/ J 2 belongs to the 0(8) uni- 
versality class. This transition is beyond the Landau's 
paradigm in the sense that the field theories (fTTj) and 
(|12[) are written in terms of spinon field instead of physi- 
cal order parameters. The physical order parameters are 
bilinears of spinon, which implies that the anomalous di- 
mension of the physical order parameters are enormous 
at this transition. 

Since the GSM is S 7 /Z 2 with fundamental group 
tt\[S 7 /Z 2 ] = Z 2 , in the condensate there are gapped vi- 
sons, which is a "7r— flux" of the "Higgsed" Z 2 gauge 
field. The disordered phase is actually a Z 2 spin liquid 
with gapped but mobile visons. This Z 2 spin liquid phase 
of SU(2) spin systems can be most conveniently visual- 
ized in the quantum dimer model (QDM) on the trian- 
gular lattice^, which by tuning the dimer flipping and 
dimer potential energy, stabilizes a gapped phase with 
Z 2 topological order and no symmetry breaking. As we 
discussed earlier, two Sp(4) particles can form a Sp(4) 
singlet, therefore the QDM for Sp(4) spin systems is ex- 
actly the same as the SU(2) spins, with also a stable Z 2 
spin liquid phase. Because the Z 2 spin liquid is a decon- 
fined phase, the excitations of the Z 2 spin liquid include 



gapped Sp(4) bosonic spinons besides the visons, If we 
start with the disordered Z 2 spin liquid state, and drive 
a transition by condensing the gapped Sp(4) spinon, the 
field theory of this transition is in the same form as (jlip . 

Since on one site there is only one particle, the particu- 
lar QDM is subject to the local constraint with one dimer 
connected to each site. This type of QDM is called odd 
QDM, since the product of Z 2 electric field around each 
site is Y\a x = — 1j which will attach a it— flux to each 
hexagon of the dual honeycomb lattice of the triangular 
lattice. This tt— flux seen by the visons will lead to four 
degenerate minima in the vison band, and the condensa- 
tion of the vison at these minima breaks the translation 
and rotation symmetry of the lattice^, and the transition 
has been suggested to be an 0(4) transition. 

In this section we discussed the transition between the 
Z 2 spin liquid and the x y/3 state of the Sp(4) spin 
system. For comparison, let us briefly discuss the order- 
disorder transition of the V3 x \/3 state in the stan- 
dard Landau theory, ignoring the topological nature of 
the Z 2 spin liquid. In the \/3 x a/3 order, both time- 
reversal and Sp(4) spin symmetries are broken. A general 
Ginzburg-Landau Lagrangian can be written in terms of 
the time-reversal even 0(5) vector n a which is defined 
as the long wavelength field of F a , and two adjoint vec- 
tors n° b and introduced in Eq. [TUJ At the quadratic 
level none of these three vectors mix, while at the cu- 
bic order a mixing term is allowed by the Sp(4) sym- 
metry: J2i=i e abcdenf h n^ d n e , this term implies that the 
ordering of the adjoint vectors would drive the order of 
n a , but the statement is not necessarily true conversely. 
If the 0(5) vector n a is ordered while the adjoint vec- 
tors nf , n 2 b are disordered, the system breaks the Sp(4) 
symmetry while preserving the time-reversal symmetry. 
Therefore if the system is tuned towards the disordered 
phase, the Landau's theory allows for multiple transi- 
tions, with the time-reversal symmetry restored before 
the Sp(4) symmetry. A uniform collinear order F a has 
GSM S 4 = SO(5)/SO(4), therefore the transition of n a 
belongs to the 0(5) universality class. The transition 
associated with time-reversal symmetry breaking is de- 
scribed by the 0(10) vectors nf and ng 6 , with various 
anisotropies in the background of the gapless 0(5) or- 
dering n a . For instance, at the quartic order, there is 
a term which imposes the "orthogonality" between the 
two 0(10) vectors: b nfnf) 2 . The nature of this 
transition requires more detailed analysis. By contrast, 
the Schwinger boson and field theory analysis show that 
there can be a direct 0(8) transition between the phase 
with coexistence of n a and n^ a h, and a spin disordered 
phase with Z 2 topological order. 



III. SP(4) HEISENBERG MODEL ON THE 
SQUARE LATTICE 

Now let us switch the gear to the square lattice. On the 
square lattice, at the point with J± = J% > 0, the model 
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([3]) can be mapped to the SU(4) Heisenberg model with 
fundamental representation on one sublattice and conju- 
gate representation on the other—. The equivalence can 
be shown by performing transformation S a — > J* S a J 
on one of the sublattices, and using the identities in 
(J2j) . This point J\ = J2 has been thoroughly studied 
by means of large- N generalizatio n 16 ! 17 ' 18 and quantum 
Monte Carloi 9 -. It is agreed that at this point the spinon 
b a condenses, and there is a small Neel momen t 19 ' 20 . 
In the Schwinger boson language, the Neel state on the 
square lattice corresponds to the condensate of Schwinger 
bosons with nonzero expectation of (Qij), which seems 
to break the U(l) gauge symmetry down to Z2. How- 
ever, the U(l) gauge symmetry can be restored if the 
Schwinger bosons on the two sublattices are associated 
with opposite gauge charges, therefore the connection be- 
tween spinon b a and low energy field z a is 

b a ~ z a , (sublattice A), 

b a ~ JapZp> (sublattice B). (14) 

The GSM of the Schwinger boson condensate is 

U(4)/[U(1) <g> U(3)] = 5 7 /U(l) = CP(3). (15) 

The field theory for this condensate is most appropriately 
described by the CP (3) model 



L = E ^ ~ ia f)*«| 2 + r\z\ 2 + g(\z\ 2 ) 2 + 



(16) 



0=1 



Again, if we perturb this field theory with Sp(4) invariant 
terms, the GSM is still CP(3), and the critical behavior 
is unchanged. The condensate of z a has staggered spin 
order T a b but uniform nematic order T a on the square 
lattice. 

In the condensate of z, gauge field is Higgsed; if 
z is disordered, a M would be in a gapless photon phase 
if the gauge fluxes are conserved. However, because 
7T2[CP(3)] = Z, the ground state manifold can have sin- 
gular objects in the 2+1 dimensional space timei 6 -, which 
corresponds to the monopole of the compact U(l) gauge 
field a^. The conservation of gauge fluxes is broken by 
the monopoles, which due to its Berry phase will drive 
the system to a phase breaking the lattice symmetr y 16 ' 21 . 

At another point with — J2 = J\ > 0, this model 
is SU(4) invariant with fundamental representations 
on both sublattices. This point is not so well stud- 
ied. A fermionic mean field theory 2 ^ and an exact 
diagonalization 2;? on a 4 x 4 lattice has been applied to 
this point, the results suggest that the ground state may 
be a plaquette order as depicted in Fig. [2l with four par- 
ticles forming a SU(4) singlet on every one out of four 
unit squares. A similar plaquette ordered phase is ob- 
tained on the spin ladder—. It is interesting to consider 
the dynamics of the plaquettes, for instance, in 3 dimen- 
sional cubic lattice, a quantum plaquette model as a gen- 
eralization of the quantum dimer model has been stud- 
ied both numerically™ and analytically 26 . If we perturb 
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FIG. 2: Fig. a, the SU(4) plaquette order pattern, four SU(4) 
particles around the colored squares form a SU(4) singlet; Fig. 
b the particular type of VBS state depends on the phase angle 
of the monopole operator. 



away from the SU(4) point with Sp(4) invariant terms, 
this plaquette order is expected to persist into a finite re- 
gion in the phase diagram due to its gapped nature. This 
phase presumably can be continuously connected to the 
Sp(4) VBS state with Sp(4) singlets resonating on every 
one of four unit squares (Fig. [5]), because both states 
are gapped and break the same lattice symmetry. More 
details about the possible phases on the square lattice is 
under study by another group^l. 

The dimer resonating plaquette state can be under- 
stood in the same way as the dimer columnar state as 
the proliferation of monopoles of the compact U(l) gauge 
field, and the oscillating Berry phase of the monopoles 
will choose the specific lattice symmetry breaking pat- 
tern. Both the dimer columnar order and the dimer 
plaquette order can be viewed as a condensate of fluxes 
of U(l) gauge field with the U(l) conservation of fluxes 
breaking down to Z4, and if the phase angle of the con- 
densate is 2n7r/4 the system is in the columnar state, 
while if the phase angle is (2n + l)ir/4 the system is in 
the dimer plaquette phased 6 - (Fig. [5]). If one considers 
a pure QDM on the square lattice, the crystalline pat- 
tern can be obtained from the dual rotor model with 
Lagrangia n 28 ' 29 : 



Ld = (<9 M x) 2 - acos(87rx). 



(17) 



Here exp(i27rx) is the monopole operator which creates 
a 27T flux of the U(l) gauge field. Now whether the sys- 
tem favors columnar order or dimer resonating plaquette 
order simply depends on the sign of a. 

Now we conjecture a phase diagram (Fig. [3]): suppose 
J\ is fixed, and we tune J2; if Ji > J c the system re- 
mains in the condensate of z, which is the Neel order of 
spin operators; when J2 < J c the system loses the Neel 
order and enters the VBS state. This transition can be a 
direct second order transition, and the field theory is de- 
scribed by the CP(3) model in (JTBJ), assuming the CP(3) 
model itself has a second order transition. The most im- 
portant instability on this field theory is the monopole of 
the compact U(l) gauge field, which is certainly relevant 



5 



IV. SUMMARY AND EXTENSION 



VBS s \ / Neel 
• ^» • *~ 

SU(4) A SU(4) B 

FIG. 3: The conjectured phase diagram for the Heisenberg 
model in equation Q. By tuning one parameter J2/J1, the 
system evolves from the Neel state to the dimerized VBS 
state, and the transition can be continuous. The red and 
blue dashed line denotes the magnitude of the Neel and VBS 
order parameter. SU(4)a is the SU(4) invariant point with 
fundamental representation on all sites; SU(4)b point is an- 
other SU(4) point with fundamental representation on one 
sublattice, and conjugate representation on the other. 

in the crystalline phase. However, it has been shown con- 
vincingly that at the 3D XY transition, the Z4 anisotropy 
of the XY variable is irrelevant, and it was also argued 
that large number of flavor of boson field tends to sup- 
press the monopole effects 4 , therefore it is likely that the 
monopoles are irrelevant at the critical point described 
by field theory (flB)) . Compared with the SU(2) spin sys- 
tem, our Sp(4) system with doubled number of complex 
boson fields has a better chance to ensure the irrelevance 
of monopole perturbations at the transition. We also 
want to point out that between the Neel order and the 
dimer resonating plaquette order, an intermediate phase 
with columnar order is also possible. But the transition 
between the Neel order and the columnar order is also 
described by field theory (|16p. and the columnar order 
is connected to the resonating plaquette order through a 
first order transition. 



In this work we studied the quantum phase transitions 
beyond the Landau's paradigm in the spin-3/2 cold atom 
systems with emergent enlarged Sp(4) symmetry. Com- 
pared with the J — Q model studied before^, the spin 
model we considered is very realistic, we propose these 
results are observable in real experimental systems in fu- 
ture. It would also been interesting to study the Heisen- 
berg model in this work through numerical techniques. 
A careful numerical study of the classical CP (3) model 
without monopoles is also desired, as has been done re- 
cently for the SU(2) invariant CP(1) model^i. 

The current work focused on the parameter regime 
with J 1 > 0. In the regime with J\ < 0, the Schwinger 
boson formalism would lead to the ordered state with 
nonzero expectation value K = (Kij), and the Schwinger 
bosons condense at momentum (0,0). This state is the 
ferromagnetic state with uniform order n ab and n a . The 
ferromagnetic state and the Neel state can be connected 
through a first order transition. More theoretical tools 
are desired to determine the other parts of the phase dia- 
gram accurately. We will leave this to the future workiS. 
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